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Abstract 

We present a search for lepton-flavor- violating r decays into three leptons (electrons 
or muons) using 782 fb"^ of data collected with the Belle detector at the KEKB 
asymmetric-energy e'^e~ collider. No evidence for these decays is observed and we 
set 90% confidence level upper limits on the branching fractions between 1.5 x 10~® 
and 2.7 x IQ-^. 



1 Introduction 



Lepton flavor violation (LFV) appears in various extensions of the Standard 
Model (SM). In particular, lepton- flavor- violating r~ — t- t'i'^t' (where I = e 
or /i ) decays are discussed in various supersymmetric models pl[2][3]|^ll5|6|7f8] . 
models with little Higgs [9|T0] . left-right symmetric models [11] as well as 
models with heavy singlet Dirac neutrinos [12] and very light pseudoscalar 
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bosons [13]. Some of these models with certain combinations of parameters 
predict that the branching fractions for r~ — )■ i't^l' decays can be as large 
as 10~^, which is in the range already accessible in high-statistics B factory 
experiments. 

Searches for lepton flavor violation in r~ — > (where i = e or fi) decays 

have been performed since 1982 [T3], starting from the pioneering experiment 
MARKII [15]. In the previous high-statistics analyses, Belle (BaBar) reached 
90% confidence level upper limits on the branching fractions of the order of 
10"*^ [TB|T7] . based on samples with about 535 (376) fb~^ of data. Here, we 
update our previous results with a larger data set (782 fb~^), collected with 
the Belle detector at the KEKB asymmetric-energy e~^e~ collider [18], taken 
at the T(4S') resonance and 60 MeV below it. We apply the same selection 
criteria as in the previous analysis, but optimized for the new data sample. 

The Belle detector is a large-solid-angle magnetic spectrometer that consists 
of a silicon vertex detector (SVD), a 50-layer central drift chamber (CDC), an 
array of aerogel threshold Cherenkov counters (ACC), a barrel-like arrange- 
ment of time-of-fiight scintillation counters (TOF), and an electromagnetic 
calorimeter comprised of CsI(Tl) crystals (ECL), all located inside a supercon- 
ducting solenoid coil that provides a 1.5 T magnetic field. An iron fiux-return 
located outside the coil is instrumented to detect mesons and to identify 
muons (KLM). The detector is described in detail elsewhere [T9] . 

Leptons are identified using likelihood ratios calculated from the response of 
various subsystems of the detector. For electron identification, the likelihood 
ratio is defined as V{e) = Ce/{Ce + Cx), where £e and Cx are the likelihoods 
for electron and non-electron hypotheses, respectively, determined using the 
ratio of the energy deposit in the ECL to the momentum measured in the 
SVD and CDC, the shower shape in the ECL, the matching between the 
position of charged track trajectory and the cluster position in the ECL, the 
hit information from the ACC and the dE/dx information in the CDC |20j . 
For muon identification, the likelihood ratio is defined as (V{fi) = C^/ {C^ + 
Lt, + £i^)), where £^ and Lk are the likelihoods for muon, pion and kaon 
hypotheses, respectively, based on the matching quahty and penetration depth 
of associated hits in the KLM [2T| . 

In order to optimize the event selection and to estimate the signal efficiency, 
we use Monte Carlo (MC) samples. The signal and the background (BG) 
events from generic r+r" decays are generated by KORALB/TAUOLA [22] • 
In the signal MC, we generate r+r" pairs, where one r decays into three 
leptons and the other r decays generically. All leptons from r~ i~t^i~ 
decays are assumed to have a phase space distribution in the r lepton's rest 
frame [23]. Other backgrounds including BB and e^e~ qq {q = u,d,s,c) 
processes, Bhabhas, e~^e~ — )■ and two-photon processes are generated 
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by EvtGen [21], BHLUMI [25], KKMC [22], and AAFHB [26], respectively. 
All kinematic variables are calculated in the laboratory frame unless otherwise 
specified. In particular, variables calculated in the e^e~ center-of-mass (CM) 
system are indicated by the superscript "CM" . 



2 Event Selection 

We search for r^r~ events in which one r decays into three leptons (signal 
r), while the other r decays into one charged track, any number of additional 
photons, and neutrinos (tag r) ^ . Candidate r-pair events are required to have 
four tracks with zero net charge. The following r~ decays into three leptons 
are searched for: e~e~^e~, fi~fi~^fi~, e~^~^fi~, ^~e~^e~, /i~e"'"/i~, and e~fi~^e~. 
Since each decay mode has a different mix of backgrounds, the event selection 
is optimized mode by mode. We optimize the selection criteria to improve the 
prospect of observing evidence of a genuine signal, rather than to minimize 
the expected upper limits, as detailed later. 

The event selection starts by reconstructing four charged tracks and any num- 
ber of photons within the fiducial volume defined by —0.866 < cos 6' < 0.956, 
where 6 is the polar angle relative to the direction opposite to that of the 
incident beam in the laboratory frame. The transverse momentum {pt) of 
each charged track and energy of each photon (E^) are required to satisfy 
the requirements pt > 0.1 GeV/c and > 0.1 GeV, respectively. For each 
charged track, the distance of the closest approach with respect to the inter- 
action point is required to be within ±0.5 cm in the transverse direction and 
within ±3.0 cm in the longitudinal direction. 

The particles in an event are then separated into two hemispheres referred to 
as the signal and tag sides using the plane perpendicular to the thrust axis, as 
calculated from the observed tracks and photon candidates [2Z] . The tag side 
contains a charged track while the signal side contains three charged tracks. 
We require all charged tracks on the signal side to be identified as leptons. 
The electron (muon) identification criteria are V{e) > 0.9 {V{n) > 0.9) for 
momenta greater than 0.3 GeV/c (0.6 GeV/c). The electron (muon) identifi- 
cation efficiency for our selection criteria is 91% (85%) while the probability 
of misidentifying a pion as an electron (muon) is below 0.5% (2%). 

To ensure that the missing particles are neutrinos rather than photons or 
charged particles that pass outside the detector acceptance, we impose addi- 
tional requirements on the missing momentum Pmiss? which is calculated by 

^ Unless otherwise stated, charge-conjugate decays are imphed throughout this pa- 
per. 
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Fig. 1. Distribution of the thrust magnitude T for the t~ — ?> ^.'^ selection. The 
points with error bars are data, and the open histogram shows the BG estimated 
by MC. The shaded histogram is the BG from e'^e~ — )> qq. The dashed histogram 
is signal MC. The region indicated by the arrow between the two vertical lines is 
selected. 

subtracting the vector sum of the momenta of all tracks and photons from 
the sum of the e+ and e~ beam momenta. We require that the magnitude of 
Pmiss be greater than 0.4 GeV/c and that its direction point into the fiducial 
volume of the detector. 



To reject qq background, the magnitude of thrust (T) should lie in the range 
0.90 < T < 0.97 for all modes except for r~ — )■ e~e~^e~ for which we require 
0.90 < T < 0.96. The T distribution for fi~fi~^fi~ is shown in Fig. [T] We 

also require 5.29 GeV < E^^ < 9.5 GeV, where E^^ is the total visible energy 
in the CM system, defined as the sum of the energies of the three leptons, the 
charged track on the tag side (with a pion mass hypothesis) and all photon 
candidates. 

Since neutrinos are emitted only on the tag side, the direction of Pmiss should 
lie within the tag side of the event. The cosine of the opening angle between 
Pmiss and the charged track on the tag side in the CM system, cos6'^j^g_n^jgg, 
is therefore required to lie in the range 0.0 < cos ^J^^^g.j^jgg < 0.98. The re- 
quirement of cos ^Jj^j^g_j^jgg < 0.98 suppresses Bhabha, and two-photon 
backgrounds since an undetected radiated photon results in a missing mo- 
mentum in the same ECL cluster as the tag-side track [28] . The reconstructed 
mass on the tag side using a charged track (with a pion mass hypothesis) and 
photons, mtag, is required to be less than 1.78 GeV/c^. 

Conversions (7 — )■ e~^e~) are a large background for the — > e~e'^e~ and 
fj,~e~^e~ modes. We require Mee > 0.2 GeV/c^, to reduce these backgrounds 
further. For the r~ e~e~^e~ and r~ e~fj,^fj,~ modes, the charged track 
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Table 1 

Selection criteria for the missing momentum (pmiss) and missing mass squared 
(m^jgg) for each mode. The units for Pmiss and rri^-^^^ are GeV/c and (GeV/c^)^, 
respectively. 



Mode 


Hadronic tag 


Leptonic tag 


T~ — )• 

— )• e^fj.^fi'^ 


Pmiss > -3.0 m^i33 - 1.0 
Pmiss > 3.0 m^^jgg - 1.5 


Pmiss > -2.5 m^igg 
Pmiss > 1-3 m^igg - 1.0 


T~ — > e~e"'"e~ 


Pmiss > -3.0 m^igg - 1.0 
Pmiss > 4.2 m^^jgg - 1.5 


Pmiss > —2.5 fTT-jniss 
Pmiss > 2.0 m^igg - 1.0 




not applied 


not applied 



on the tag side is required not to be an electron. We apply the requirement 
V{e) < 0.1 since a large background from two-photon and Bhabha events 
still remains. Furthermore, we reject the event if the projection of the charged 
track on the tag side is in gaps between the ECL barrel and endcap. To 
reduce backgrounds from Bhabha and fi^fi~ events with extra tracks due to 
interaction with the detector material, we require that the momentum in the 
CM system of the charged track on the tag side be less than 4.5 GeV/c for 
the T~ — >■ e~e~^e~ and r~ — )■ ^~e~^e~ modes. 

Finally, to suppress backgrounds from generic t~^t~ and qq events, we apply a 
selection based on the magnitude of the missing momentum Pmiss and missing 
mass squared m^^gg for all modes except for r~ — )■ e^/i"//" and /i"'"e~e~. We 
do not apply this requirement for these two modes since their backgrounds 
are much smaller. We apply different selection criteria depending on whether 
the r decay on the tag side is hadronic or leptonic: the number of emitted 
neutrinos is two (one) when the r decay on the tag side is leptonic (hadronic). 
Therefore, we separate events into two classes according to the track on the 
tag side: leptonic or hadronic. The selection criteria are listed in Table [H 

For the optimization, we examine the relation between the number of events 
(^obs.)' which would need to be observed to obtain 99% confidence level (CL) 
evidence, and the number of expected BG events (A^bg)- We find that better 
sensitivity is obtained for smaller A^bg? provided that the signal efficiency does 
not drop drastically. For example, when we reduce N^g from 1 to 0.1, A^^^g 
decreases from 5 to 2, as calculated with the POLE program [22]. This is 
equivalent to an improvement of the effective efficiency by a factor of 2.5. 

For the case of r — > /i/i/i we obtain an expected BG of 0.13 ± 0.06 with an 
efficiency of 7.6% for the event selection described above. In this case, the 
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branching fraction obtained from A^o^s. is -B99 = 1.8 x 10~^, and the upper 
hmit for the branching fraction at the 90% CL is i5^J" < 2.1 x 10~^ for zero 
observed events. When we relax the selection criteria by removing the require- 
ments on Pmiss-''^miss' momcutum and mass of the tag side, cos9^^_^^gg, 
thrust, and so on, we obtain an expected BG of 0.42±0.17 with an efficiency of 
8.9%, so that ,899 = 2.3 x 10~* and B^q < 1.6 x 10^^ for zero observed events. 
For the relaxed selection criteria, in the Feldman- Cousins approach [30] the 
upper limits on the branching fractions are small when the number of observed 
events fluctuates below the number of expected background events. As men- 
tioned above, we optimize the selection criteria to obtain good sensitivity for 
signal discovery. Therefore, we choose the selection criteria described above to 
minimize Bgg with the signal region defined below. 

The following main background sources remain after the event selection, which 
are estimated from the MC and data: Bhabha and 77 — )■ e^e~ for r~ 
e~e~^e~, 77 — fi~^fi~ for r~ /i~e^e~ and e~fi'^fi~ , r-pairs and qq for r~ — )■ 
/i"//"*"/!", e~/i"'"e~ and ^~e~^fi^. 



3 Signal and Background Estimation 



The signal candidates are examined in two-dimensional plots of the 
invariant mass [M^i), and the difference between the summed energy and the 
beam energy in the CM system (AE). A signal event should have M^i close 
to the r-lepton mass and AE close to zero. We define an elliptical signal 
region in the M^^-AE plane, which is optimized using the signal MC, to have 
a minimum area containing 90% of the signal after all the selections. 

In order not to bias our choice of selection criteria, we blind the data in the 
signal region and estimate the signal efficiency and the number of background 
events from the MC and the data outside the signal region. Figure [2] shows 
scatter-plots for the data and the signal MC distributed over ±20cr on the 
M-^i — AE plane. We observe no events for r~ — )■ ^~ e~ , e~ fi~^ e~ , fj,~ e'^ fj,~ , 
one event for r~ fi~fi'^fi~, two events for r~ — > e~e~^e~ and three events 
for e~fi~^fi~, outside the signal region. The 7 conversion veto effectively 

reduces the background for r~ — )• e~e~^e~ . 

The final estimate of the number of background events is based on the data 
with looser selection criteria for particle identification and the event selection 
in the M^^ sideband region, which is defined as the box inside the horizontal 
lines but excluding the signal region, as shown by the horizontal lines in Fig. [2l 
For example, we obtain 5 events in the sideband region for r~ — >■ fi~^~^fi~, 
when a less stringent PID criterion, V{fi) > 0.6, is applied. Assuming that 
the background distribution is uniform in the sideband region, the number of 
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Table 2 

Results with nominal selection criteria: the signal efficiency (e), the number of 
expected background events (A'bg) estimated from the sideband data, the total 
systematic uncertainty (fjsyst)) the number of observed events in the signal region 
(Nohs) and 90% C.L. upper limit on the branching fraction (B) for each individual 
mode. 



Mode 


ei%) 


Nbg 


O-syst (%) 


iVobs 






— )• e~e~^e~ 


6.0 


0.21±0.15 


9.8 





<2.7 


T~ 


— )■ fj,~fi~^fi~ 


7.6 


0.13±0.06 


7.4 





<2.1 


T~ 


— )• e~ fi^f-i^ 


6.1 


0.10±0.04 


9.5 





<2.7 


T~ 


— )■ /i~e+e~ 


9.3 


0.04±0.04 


7.8 





<1.8 


T~ 


— e"*~/i^/i^ 


10.1 


0.02±0.02 


7.6 





<1.7 




— )■ ^^e^e" 


11.5 


O.OlibO.Ol 


7.7 





<1.5 



background events in the signal box is estimated by interpolating the number 
of observed events in the sideband region into the signal region. The signal 
efficiency and the number of expected background events for each mode are 
summarized in Table O 

We estimate the systematic uncertainties due to lepton identification, charged 
track finding, MC statistics, and the integrated luminosity. The uncertainty 
due to the trigger efficiency is negligible compared with the other uncertain- 
ties. The uncertainties due to lepton identification are 2.2% per electron and 
2.0% per muon. The uncertainty due to charged track finding is estimated 
to be 1.0% per charged track. The uncertainty due to the electron veto on 
the tag side appUed for the r~ — )■ e~e~^e~ and r~ — >• e~fi'^n~ modes is es- 
timated to be the same as the uncertainty due to the electron identification. 
For other modes, we use the same systematic uncertainty for leptonic and 
hadronic decays on the tag side, because we do not apply any lepton/hadron 
identification requirements for any charged track on the tag side. The uncer- 
tainties due to MC statistics and luminosity are estimated to be (0.5 - 0.9)% 
and 1.4%, respectively. We do not include an uncertainty due to the signal 
MC model. All these uncertainties are added in quadrature, and the total 
systematic uncertainty for each mode is listed in Table [2j 



4 Upper Limits on the branching fractions 

Finally, we examine the signal region and find no events for all considered 
modes. Therefore, we set upper limits on the branching fractions of r~ — )■ 
based on the Feldman- Cousins method. The 90% C.L. upper limit 
on the number of signal events including the systematic uncertainty (sgo) is 
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Fig. 2. Scatter-plots in the M^i - AE plane, showing the ±20(T area for (a) 
T~ — )• e~e'^e~, (b) r~ — > ij,~fi~^n~, (c) r~ ^ e~ , (d) r~ ^ /i~e+e~, (e) 
r~ ^ e+/i~/u~ and (f) r~ ^ /i+e~e~. The data are indicated by the sohd circles. 
The filled boxes show the MC signal distribution with arbitrary normalization. The 
elliptical signal regions shown by the solid curves are used for evaluating the signal 
yield. The region between the horizontal solid lines excluding the signal region is 
used to estimate the background expected in the elliptical region. 



obtained by the POLE program without conditioning [29] with the number of 
expected background events, the number of observed events and the systematic 
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uncertainty. The upper limit on the branching fraction (B) is then given by 

e(,-^r«-)<^, (1) 

where the number of r pairs, Nrr = 719 x 10^, is obtained from the integrated 
luminosity of 782 fb~^ and the cross section of r pair production, which is 
calculated in the updated version of KKMC |3l] to be a^r = (0.919 ± 0.003) 
nb. The 90% CL. upper limits on the branching fractions B{t^ i~i~^£~) 
are in the range between 1.5 x 10~® and 2.7 x 10"*^ and are summarized in 
Table H 



5 Summary 

We report results of a search for lepton-fiavor-violating r decays into three 
leptons using 782 fb~^ of data. No events are observed and we set 90% CL. up- 
per limits on the branching fractions: B{t~ — >• e~e~^e~) < 2.7 x 10"*^, B{t~ — )■ 
< 2.1 X 10-^ B{t- e-/iV) < 2.7 x 10-^ B{t- ^u-e+e") < 
1.8 x 10-^ B{t- e+fL-fi-) < 1.7 X 10-s and B{t~ /i+e-e") < 1.5 x 10^^ 
These results improve the best previously published upper limits by factors 
from 1.3 to 1.6, and are the most stringent upper limits of all r decays. These 
upper limits can be used to constrain the space of parameters in various models 
beyond the SM. 
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